Development and validation of a novel model to predict recurrence-free survival and melanoma-specific survival after sentinel lymph node biopsy in patients with melanoma: an international, retrospective, multicentre analysis.

BACKGROUND: The introduction of adjuvant systemic treatment for patients with high-risk melanomas necessitates accurate staging of disease. However, inconsistencies in outcomes exist between disease stages as defined by the American Joint Committee on Cancer (8th edition). We aimed to develop a tool to predict patient-specific outcomes in people with melanoma rather than grouping patients according to disease stage. METHODS: Patients older than 13 years with confirmed primary melanoma who underwent sentinel lymph node biopsy (SLNB) between Oct 29, 1997, and Nov 11, 2013, at four European melanoma centres (based in Berlin, Germany; Amsterdam and Rotterdam, the Netherlands; and Warsaw, Poland) were included in the development cohort. Potential predictors of recurrence-free and melanoma-specific survival assessed were sex, age, presence of ulceration, primary tumour location, histological subtype, Breslow thickness, sentinel node status, number of sentinel nodes removed, maximum diameter of the largest sentinel node metastasis, and Dewar classification. A prognostic model and nomogram were developed to predict 5-year recurrence-free survival on a continuous scale in patients with stage pT1b or higher melanomas. This model was also calibrated to predict melanoma-specific survival. Model performance was assessed by discrimination (area under the time-dependent receiver operating characteristics curve [AUC]) and calibration. External validation was done in a cohort of patients with primary melanomas who underwent SLNB between Jan 30, 1997, and Dec 12, 2013, at the Melanoma Institute Australia (Sydney, NSW, Australia). FINDINGS: The development cohort consisted of 4071 patients, of whom 2075 (51%) were female and 1996 (49%) were male. 889 (22%) had sentinel node-positive disease and 3182 (78%) had sentinel node-negative disease. The validation cohort comprised 4822 patients, of whom 1965 (41%) were female and 2857 (59%) were male. 891 (18%) had sentinel node-positive disease and 3931 (82%) had sentinel node-negative disease. Median follow-up was 4·8 years (IQR 2·3-7·8) in the development cohort and 5·0 years (2·2-8·9) in the validation cohort. In the development cohort, 5-year recurrence-free survival was 73·5% (95% CI 72·0-75·1) and 5-year melanoma-specific survival was 86·5% (85·3-87·8). In the validation cohort, the corresponding estimates were 66·1% (64·6-67·7) and 83·3% (82·0-84·6), respectively. The final model contained six prognostic factors: sentinel node status, Breslow thickness, presence of ulceration, age at SLNB, primary tumour location, and maximum diameter of the largest sentinel node metastasis. In the development cohort, for the model's prediction of recurrence-free survival, the AUC was 0·80 (95% CI 0·78-0·81); for prediction of melanoma-specific survival, the AUC was 0·81 (0·79-0·84). External validation showed good calibration for both outcomes, with AUCs of 0·73 (0·71-0·75) and 0·76 (0·74-0·78), respectively. INTERPRETATION: Our prediction model and nomogram accurately predicted patient-specific risk probabilities for 5-year recurrence-free and melanoma-specific survival. These tools could have important implications for clinical decision making when considering adjuvant treatments in patients with high-risk melanomas. FUNDING: Erasmus Medical Centre Cancer Institute.

Personalized decision-making for aneurysm treatment of aneurysmal subarachnoid hemorrhage: development and validation of a clinical prediction tool.

BACKGROUND: In patients with aneurysmal subarachnoid hemorrhage suitable for endovascular coiling and neurosurgical clip-reconstruction, the aneurysm treatment decision-making process could be improved by considering heterogeneity of treatment effect and durability of treatment. We aimed to develop and validate a tool to predict individualized treatment benefit of endovascular coiling compared to neurosurgical clip-reconstruction. METHODS: We used randomized data (International Subarachnoid Aneurysm Trial, n = 2143) to develop models to predict 2-month functional outcome and to predict time-to-rebleed-or-retreatment. We modeled for heterogeneity of treatment effect by adding interaction terms of treatment with prespecified predictors and with baseline risk of the outcome. We predicted outcome with both treatments and calculated absolute treatment benefit. We described the patient characteristics of patients with ≥ 5% point difference in the predicted probability of favorable functional outcome (modified Rankin Score 0-2) and of no rebleed or retreatment within 10 years. Model performance was expressed with the c-statistic and calibration plots. We performed bootstrapping and leave-one-cluster-out cross-validation and pooled cluster-specific c-statistics with random effects meta-analysis. RESULTS: The pooled c-statistics were 0.72 (95% CI: 0.69-0.75) for the prediction of 2-month favorable functional outcome and 0.67 (95% CI: 0.63-0.71) for prediction of no rebleed or retreatment within 10 years. We found no significant interaction between predictors and treatment. The average predicted benefit in favorable functional outcome was 6% (95% CI: 3-10%) in favor of coiling, but 11% (95% CI: 9-13%) for no rebleed or retreatment in favor of clip-reconstruction. 134 patients (6%), young and in favorable clinical condition, had negligible functional outcome benefit of coiling but had a ≥ 5% point benefit of clip-reconstruction in terms of durability of treatment. CONCLUSIONS: We show that young patients in favorable clinical condition and without extensive vasospasm have a negligible benefit in functional outcome of endovascular coiling - compared to neurosurgical clip-reconstruction - while at the same time having a substantially lower probability of retreatment or rebleeding from neurosurgical clip-reconstruction - compared to endovascular coiling. The SHARP prediction tool ( https://sharpmodels.shinyapps.io/sharpmodels/ ) could support and incentivize a multidisciplinary discussion about aneurysm treatment decision-making by providing individualized treatment benefit estimates.

Prevention of incisional hernia with prophylactic onlay and sublay mesh reinforcement vs. primary suture only in midline laparotomies (PRIMA): long-term outcomes of a multicentre, double-blind, randomised controlled trial.

Background: Incisional hernia occurs approximately in 40% of high-risk patients after midline laparotomy. Prophylactic mesh placement has shown promising results, but long-term outcomes are needed. The present study aimed to assess the long-term incisional hernia rates of the previously conducted PRIMA trial with radiological follow-up. Methods: In the PRIMA trial, patients with increased risk of incisional hernia formation (AAA or BMI ≥27 kg/m2) were randomised in a 1:2:2 ratio to primary suture, onlay mesh or sublay mesh closure in three different countries in eleven institutions. Incisional hernia during follow-up was diagnosed by any of: CT, ultrasound and physical examination, or during surgery. Assessors and patients were blinded until 2-year follow-up. Time-to-event analysis according to intention-to-treat principle was performed with the Kaplan-Meier method and Cox proportional hazard models. Trial registration: NCT00761475 (ClinicalTrials.gov). Findings: Between 2009 and 2012, 480 patients were randomized: 107 primary suture, 188 onlay mesh and 185 sublay mesh. Five-year incisional hernia rates were 53.4% (95% CI: 40.4-64.8), 24.7% (95% CI: 12.7-38.8), 29.8% (95% CI: 17.9-42.6), respectively. Compared to primary suture, onlay mesh (HR: 0.390, 95% CI: 0.248-0.614, p < 0.001) and sublay mesh (HR: 0.485, 95% CI: 0.309-0.761, p = 0.002) were associated with a significantly lower risk of incisional hernia development. Interpretation: Prophylactic mesh placement remained effective in reducing incisional hernia occurrence after midline laparotomy in high-risk patients during long-term follow-up. Hernia rates in the primary suture group were higher than previously anticipated. Funding: B. Braun.

Can Machine Learning Aid the Selection of Percutaneous vs Surgical Revascularization?

BACKGROUND: In patients with 3-vessel coronary artery disease (CAD) and/or left main CAD, individual risk prediction plays a key role in deciding between percutaneous coronary intervention (PCI) and coronary artery bypass grafting (CABG). OBJECTIVES: The aim of this study was to assess whether these individualized revascularization decisions can be improved by applying machine learning (ML) algorithms and integrating clinical, biological, and anatomical factors. METHODS: In the SYNTAX (Synergy between PCI with Taxus and Cardiac Surgery) study, ML algorithms (Lasso regression, gradient boosting) were used to develop a prognostic index for 5-year death, which was combined, in the second stage, with assigned treatment (PCI or CABG) and prespecified effect-modifiers: disease type (3-vessel or left main CAD) and anatomical SYNTAX score. The model's discriminative ability to predict the risk of 5-year death and treatment benefit between PCI and CABG was cross-validated in the SYNTAX trial (n = 1,800) and externally validated in the CREDO-Kyoto (Coronary REvascularization Demonstrating Outcome Study in Kyoto) registry (n = 7,362), and then compared with the original SYNTAX score II 2020 (SSII-2020). RESULTS: The hybrid gradient boosting model performed best for predicting 5-year all-cause death with C-indexes of 0.78 (95% CI: 0.75-0.81) in cross-validation and 0.77 (95% CI: 0.76-0.79) in external validation. The ML models discriminated 5-year mortality better than the SSII-2020 in the external validation cohort and identified heterogeneity in the treatment benefit of CABG vs PCI. CONCLUSIONS: An ML-based approach for identifying individuals who benefit from CABG or PCI is feasible and effective. Implementation of this model in health care systems-trained to collect large numbers of parameters-may harmonize decision making globally. (Synergy Between PCI With TAXUS and Cardiac Surgery: SYNTAX Extended Survival [SYNTAXES]; NCT03417050; SYNTAX Study: TAXUS Drug-Eluting Stent Versus Coronary Artery Bypass Surgery for the Treatment of Narrowed Arteries; NCT00114972).

Integrated care in patients with atrial fibrillation- a predictive heterogeneous treatment effect analysis of the ALL-IN trial.

INTRODUCTION: Integrated care is effective in reducing all-cause mortality in patients with atrial fibrillation (AF) in primary care, though time and resource intensive. The aim of the current study was to assess whether integrated care should be directed at all AF patients equally. METHODS: The ALL-IN trial (n = 1,240 patients, median age 77 years) was a cluster-randomized trial in which primary care practices were randomized to provide integrated care or usual care to AF patients aged 65 years and older. Integrated care comprised of (i) anticoagulation monitoring, (ii) quarterly checkups and (iii) easy-access consultation with cardiologists. For the current analysis, cox proportional hazard analysis with all clinical variables from the CHA2DS2-VASc score was used to predict all-cause mortality in the ALL-IN trial. Subsequently, the hazard ratio and absolute risk reduction were plotted as a function of this predicted mortality risk to explore treatment heterogeneity. RESULTS: Under usual care, after a median of 2 years follow-up the absolute risk of all-cause mortality in the highest-risk quarter was 31.0%, compared to 4.6% in the lowest-risk quarter. On the relative scale, there was no evidence of treatment heterogeneity (p for interaction = 0.90). However, there was substantial treatment heterogeneity on the absolute scale: risk reduction in the lowest risk- quarter of risk 3.3% (95% CI -0.4% - 7.0) compared to 12.0% (95% CI 2.7% - 22.0) in the highest risk quarter. CONCLUSION: While the relative degree of benefit from integrated AF care is similar in all patients, patients with a high all-cause mortality risk have a greater benefit on an absolute scale and should therefore be prioritized when implementing integrated care.

Personalised decision making to predict absolute metastatic risk in cutaneous squamous cell carcinoma: development and validation of a clinico-pathological model.

Background: Cutaneous squamous cell carcinoma (cSCC) is a common skin cancer, affecting more than 2 million people worldwide yearly and metastasising in 2-5% of patients. However, current clinical staging systems do not provide estimates of absolute metastatic risk, hence missing the opportunity for more personalised treatment advice. We aimed to develop a clinico-pathological model that predicts the probability of metastasis in patients with cSCC. Methods: Nationwide cohorts from (1) all patients with a first primary cSCC in The Netherlands in 2007-2008 and (2) all patients with a cSCC in 2013-2015 in England were used to derive nested case-control cohorts. Pathology records of primary cSCCs that originated a loco-regional or distant metastasis were identified, and these cSCCs were matched to primary cSCCs of controls without metastasis (1:1 ratio). The model was developed on the Dutch cohort (n = 390) using a weighted Cox regression model with backward selection and validated on the English cohort (n = 696). Model performance was assessed using weighted versions of the C-index, calibration metrics, and decision curve analysis; and compared to the Brigham and Women's Hospital (BWH) and the American Joint Committee on Cancer (AJCC) staging systems. Members of the multidisciplinary Skin Cancer Outcomes (SCOUT) consortium were surveyed to interpret metastatic risk cutoffs in a clinical context. Findings: Eight out of eleven clinico-pathological variables were selected. The model showed good discriminative ability, with an optimism-corrected C-index of 0.80 (95% Confidence interval (CI) 0.75-0.85) in the development cohort and a C-index of 0.84 (95% CI 0.81-0.87) in the validation cohort. Model predictions were well-calibrated: the calibration slope was 0.96 (95% CI 0.76-1.16) in the validation cohort. Decision curve analysis showed improved net benefit compared to current staging systems, particularly for thresholds relevant for decisions on follow-up and adjuvant treatment. The model is available as an online web-based calculator (https://emc-dermatology.shinyapps.io/cscc-abs-met-risk/). Interpretation: This validated model assigns personalised metastatic risk predictions to patients with cSCC, using routinely reported histological and patient-specific risk factors. The model can empower clinicians and healthcare systems in identifying patients with high-risk cSCC and offering personalised care/treatment and follow-up. Use of the model for clinical decision-making in different patient populations must be further investigated. Funding: PPP Allowance made available by Health-Holland, Top Sector Life Sciences & Health, to stimulate public-private partnerships.

External validation of 18 F-FDG PET-based radiomic models on identification of residual oesophageal cancer after neoadjuvant chemoradiotherapy.

OBJECTIVES: Detection of residual oesophageal cancer after neoadjuvant chemoradiotherapy (nCRT) is important to guide treatment decisions regarding standard oesophagectomy or active surveillance. The aim was to validate previously developed 18 F-FDG PET-based radiomic models to detect residual local tumour and to repeat model development (i.e. 'model extension') in case of poor generalisability. METHODS: This was a retrospective cohort study in patients collected from a prospective multicentre study in four Dutch institutes. Patients underwent nCRT followed by oesophagectomy between 2013 and 2019. Outcome was tumour regression grade (TRG) 1 (0% tumour) versus TRG 2-3-4 (≥1% tumour). Scans were acquired according to standardised protocols. Discrimination and calibration were assessed for the published models with optimism-corrected AUCs >0.77. For model extension, the development and external validation cohorts were combined. RESULTS: Baseline characteristics of the 189 patients included [median age 66 years (interquartile range 60-71), 158/189 male (84%), 40/189 TRG 1 (21%) and 149/189 (79%) TRG 2-3-4] were comparable to the development cohort. The model including cT stage plus the feature 'sum entropy' had best discriminative performance in external validation (AUC 0.64, 95% confidence interval 0.55-0.73), with a calibration slope and intercept of 0.16 and 0.48 respectively. An extended bootstrapped LASSO model yielded an AUC of 0.65 for TRG 2-3-4 detection. CONCLUSION: The high predictive performance of the published radiomic models could not be replicated. The extended model had moderate discriminative ability. The investigated radiomic models appeared inaccurate to detect local residual oesophageal tumour and cannot be used as an adjunct tool for clinical decision-making in patients.

Number of life-years lost at the time of diagnosis and several years post-diagnosis in patients with solid malignancies: a population-based study in the Netherlands, 1989-2019.

Background: Loss of life expectancy (LOLE) may provide more intuitive information on the impact of cancer than relative survival over a fixed time horizon (e.g., 5-year relative survival). We aimed to assess the evolution of the LOLE using a nationwide, population-based cohort including patients diagnosed with one of 17 most frequent solid malignancies. Methods: From the Netherlands Cancer Registry, we selected adult patients diagnosed with one of the 17 most frequent solid malignancies in the Netherlands during 1989-2019, with survival follow-up until 2022. We used flexible parametric survival models to estimate the LOLE at diagnosis and the LOLE after surviving several years post-diagnosis (conditional LOLE; CLOLE) by cancer type, calendar year, age, sex, and disease stage. Findings: For all cancers combined, the LOLE consistently decreased from 1989 to 2019. This decrease was most pronounced for males with prostate cancer (e.g., from 6.9 [95% confidence interval [CI], 6.7-7.1] to 2.7 [95% CI, 2.5-3.0] for 65-year-olds) and females with breast cancer (e.g., from 6.6 [95% CI, 6.4-6.7] to 1.9 [95% CI, 1.8-2.0] for 65-year-olds). The LOLE among patients with cancers of the head and neck or the central nervous system remained constant over time. Overall, the CLOLE showed that the life years lost among patients with cancer decreased with each additional year survived post-diagnosis. For example, the LOLE at diagnosis for 65-year-old females diagnosed with breast cancer in 2019 was 1.9 [95% CI, 1.8-2.0] compared with 1.7 [95% CI, 1.6-1.8], 1.0 [95% CI, 0.9-1.1], and 0.5 [95% CI, 0.5-0.6] when surviving one, five, and ten years post-diagnosis, respectively. Estimates for other combinations of patient and tumour characteristics are available in a publicly available web-based application. Interpretation: Our findings suggested that the evolution of LOLE substantially varies across cancer type, age, and disease stage. LOLE estimates help patients better understand the impact of their specific cancer diagnosis on their life expectancy. Funding: None.

Prognostic Models for Global Functional Outcome and Post-Concussion Symptoms Following Mild Traumatic Brain Injury: A Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) Study.

After mild traumatic brain injury (mTBI), a substantial proportion of individuals do not fully recover on the Glasgow Outcome Scale Extended (GOSE) or experience persistent post-concussion symptoms (PPCS). We aimed to develop prognostic models for the GOSE and PPCS at 6 months after mTBI and to assess the prognostic value of different categories of predictors (clinical variables; questionnaires; computed tomography [CT]; blood biomarkers). From the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) study, we included participants aged 16 or older with Glasgow Coma Score (GCS) 13-15. We used ordinal logistic regression to model the relationship between predictors and the GOSE, and linear regression to model the relationship between predictors and the Rivermead Post-concussion Symptoms Questionnaire (RPQ) total score. First, we studied a pre-specified Core model. Next, we extended the Core model with other clinical and sociodemographic variables available at presentation (Clinical model). The Clinical model was then extended with variables assessed before discharge from hospital: early post-concussion symptoms, CT variables, biomarkers, or all three categories (extended models). In a subset of patients mostly discharged home from the emergency department, the Clinical model was extended with 2-3-week post-concussion and mental health symptoms. Predictors were selected based on Akaike's Information Criterion. Performance of ordinal models was expressed as a concordance index (C) and performance of linear models as proportion of variance explained (R2). Bootstrap validation was used to correct for optimism. We included 2376 mTBI patients with 6-month GOSE and 1605 patients with 6-month RPQ. The Core and Clinical models for GOSE showed moderate discrimination (C = 0.68 95% confidence interval 0.68 to 0.70 and C = 0.70[0.69 to 0.71], respectively) and injury severity was the strongest predictor. The extended models had better discriminative ability (C = 0.71[0.69 to 0.72] with early symptoms; 0.71[0.70 to 0.72] with CT variables or with blood biomarkers; 0.72[0.71 to 0.73] with all three categories). The performance of models for RPQ was modest (R2 = 4% Core; R2 = 9% Clinical), and extensions with early symptoms increased the R2 to 12%. The 2-3-week models had better performance for both outcomes in the subset of participants with these symptoms measured (C = 0.74 [0.71 to 0.78] vs. C = 0.63[0.61 to 0.67] for GOSE; R2 = 37% vs. 6% for RPQ). In conclusion, the models based on variables available before discharge have moderate performance for the prediction of GOSE and poor performance for the prediction of PPCS. Symptoms assessed at 2-3 weeks are required for better predictive ability of both outcomes. The performance of the proposed models should be examined in independent cohorts.

A standardized framework for risk-based assessment of treatment effect heterogeneity in observational healthcare databases.

Treatment effects are often anticipated to vary across groups of patients with different baseline risk. The Predictive Approaches to Treatment Effect Heterogeneity (PATH) statement focused on baseline risk as a robust predictor of treatment effect and provided guidance on risk-based assessment of treatment effect heterogeneity in a randomized controlled trial. The aim of this study is to extend this approach to the observational setting using a standardized scalable framework. The proposed framework consists of five steps: (1) definition of the research aim, i.e., the population, the treatment, the comparator and the outcome(s) of interest; (2) identification of relevant databases; (3) development of a prediction model for the outcome(s) of interest; (4) estimation of relative and absolute treatment effect within strata of predicted risk, after adjusting for observed confounding; (5) presentation of the results. We demonstrate our framework by evaluating heterogeneity of the effect of thiazide or thiazide-like diuretics versus angiotensin-converting enzyme inhibitors on three efficacy and nine safety outcomes across three observational databases. We provide a publicly available R software package for applying this framework to any database mapped to the Observational Medical Outcomes Partnership Common Data Model. In our demonstration, patients at low risk of acute myocardial infarction receive negligible absolute benefits for all three efficacy outcomes, though they are more pronounced in the highest risk group, especially for acute myocardial infarction. Our framework allows for the evaluation of differential treatment effects across risk strata, which offers the opportunity to consider the benefit-harm trade-off between alternative treatments.

Estimating individualized treatment effects from randomized controlled trials: a simulation study to compare risk-based approaches.

BACKGROUND: Baseline outcome risk can be an important determinant of absolute treatment benefit and has been used in guidelines for "personalizing" medical decisions. We compared easily applicable risk-based methods for optimal prediction of individualized treatment effects. METHODS: We simulated RCT data using diverse assumptions for the average treatment effect, a baseline prognostic index of risk, the shape of its interaction with treatment (none, linear, quadratic or non-monotonic), and the magnitude of treatment-related harms (none or constant independent of the prognostic index). We predicted absolute benefit using: models with a constant relative treatment effect; stratification in quarters of the prognostic index; models including a linear interaction of treatment with the prognostic index; models including an interaction of treatment with a restricted cubic spline transformation of the prognostic index; an adaptive approach using Akaike's Information Criterion. We evaluated predictive performance using root mean squared error and measures of discrimination and calibration for benefit. RESULTS: The linear-interaction model displayed optimal or close-to-optimal performance across many simulation scenarios with moderate sample size (N = 4,250; ~ 785 events). The restricted cubic splines model was optimal for strong non-linear deviations from a constant treatment effect, particularly when sample size was larger (N = 17,000). The adaptive approach also required larger sample sizes. These findings were illustrated in the GUSTO-I trial. CONCLUSIONS: An interaction between baseline risk and treatment assignment should be considered to improve treatment effect predictions.

US and Dutch Perspectives on the Use of COVID-19 Clinical Prediction Models: Findings from a Qualitative Analysis.

INTRODUCTION: Clinical prediction models (CPMs) for coronavirus disease 2019 (COVID-19) may support clinical decision making, treatment, and communication. However, attitudes about using CPMs for COVID-19 decision making are unknown. METHODS: Online focus groups and interviews were conducted among health care providers, survivors of COVID-19, and surrogates (i.e., loved ones/surrogate decision makers) in the United States and the Netherlands. Semistructured questions explored experiences about clinical decision making in COVID-19 care and facilitators and barriers for implementing CPMs. RESULTS: In the United States, we conducted 4 online focus groups with 1) providers and 2) surrogates and survivors of COVID-19 between January 2021 and July 2021. In the Netherlands, we conducted 3 focus groups and 4 individual interviews with 1) providers and 2) surrogates and survivors of COVID-19 between May 2021 and July 2021. Providers expressed concern about CPM validity and the belief that patients may interpret CPM predictions as absolute. They described CPMs as potentially useful for resource allocation, triaging, education, and research. Several surrogates and people who had COVID-19 were not given prognostic estimates but believed this information would have supported and influenced their decision making. A limited number of participants felt the data would not have applied to them and that they or their loved ones may not have survived, as poor prognosis may have suggested withdrawal of treatment. CONCLUSIONS: Many providers had reservations about using CPMs for people with COVID-19 due to concerns about CPM validity and patient-level interpretation of the outcome predictions. However, several people who survived COVID-19 and their surrogates indicated that they would have found this information useful for decision making. Therefore, information provision may be needed to improve provider-level comfort and patient and surrogate understanding of CPMs. HIGHLIGHTS: While clinical prediction models (CPMs) may provide an objective means of assessing COVID-19 prognosis, provider concerns about CPM validity and the interpretation of CPM predictions may limit their clinical use.Providers felt that CPMs may be most useful for resource allocation, triage, research, or educational purposes for COVID-19.Several survivors of COVID-19 and their surrogates felt that CPMs would have been informative and may have aided them in making COVID-19 treatment decisions, while others felt the data would not have applied to them.

Impact of waiting time on post-transplant survival for recipients with hepatocellular carcinoma: A natural experiment randomized by blood group.

Background & Aims: When listing for liver transplantation, one can transplant as soon as possible or introduce a test-of-time to better select patients, as the tumor's biological behavior is observed. Knowing the degree of harm caused by time itself is essential to advise patients and decide on the maximum duration of the test-of-time. Therefore, we investigated the causal effect of waiting time on post-transplant survival for patients with hepatocellular carcinoma. Methods: We analyzed the UNOS-OPTN dataset and exploited a natural experiment created by blood groups. Relations between variables and assumptions were described in a causal graph. Selection bias was addressed by inverse probability weighting. Confounding was avoided using instrumental variable analysis, with an additive hazards model in the second stage. The causal effect was evaluated by estimating the difference in 5-year overall survival if all patients waited 2 months instead of 12 months. Upper bounds of the test-of-time were evaluated for probable scenarios by means of simulation. Results: The F-statistic of the first stage was 86.3. The effect of waiting 12 months vs. 2 months corresponded with a drop in overall survival rate of 5.07% (95% CI 0.277-9.69) and 8.33% (95% CI 0.47-15.60) at 5- and 10-years post-transplant, respectively. The median survival dropped by 3.41 years from 16.21 years (95% CI 15.98-16.60) for those waiting 2 months to 12.80 years (95% CI 10.72-15.90) for those waiting 12 months. Conclusions: From a patient's perspective, the choice between ablate-and-wait vs. immediate transplantation is in favor of immediate transplantation. From a policy perspective, the extra waiting time can be used to increase the utility of scarce donor livers. However, the duration of the test-of-time is bounded, and it likely should not exceed 8 months. Impact and implications: When listing patients with liver cancer for transplantation, it is unclear whether a test-of-time or immediate transplantation offer better outcomes at the population level. In this study, we found that increased liver transplant waiting times are detrimental in patients with liver cancer. Furthermore, our simulation showed that a pre-operative observational period can be useful to ensure good donor liver allocation, but that its duration should not exceed 8 months.

Endovascular versus neurosurgical aneurysm treatment: study protocol for the development and validation of a clinical prediction tool for individualised decision making.

INTRODUCTION: Treatment decisions for aneurysmal subarachnoid haemorrhage patients should be supported by individualised predictions of the effects of aneurysm treatment. We present a study protocol and analysis plan for the development and external validation of models to predict benefit of neurosurgical versus endovascular aneurysm treatment on functional outcome and durability of treatment. METHODS AND ANALYSIS: We will use data from the International Subarachnoid Aneurysm Trial for model development. The outcomes are functional outcome, measured with modified Rankin Scale at 12 months, and any retreatment or rebleed of the target aneurysm during follow-up. We will develop an ordinal logistic regression model and Cox regression model, considering age, World Federation of Neurological Surgeons grade, Fisher grade, vasospasm at presentation, aneurysm lumen size, aneurysm neck size, aneurysm location and time-to-aneurysm-treatment as predictors. We will test for interactions with treatment and with baseline risk and derive individualised predicted probabilities of treatment benefit. A benefit of ≥5% will be considered clinically relevant. Discriminative performance of the outcome predictions will be assessed with the c-statistic. Calibration will be assessed with calibration plots. Discriminative performance of the benefit predictions will be assessed with the c-for benefit. We will assess internal validity with bootstrapping and external validity with leave-one-out internal-external cross-validation. ETHICS AND DISSEMINATION: The medical ethical research committee of the Erasmus MC University Medical Center Rotterdam approved the study protocol under the exemption category and waived the need for written informed consent (MEC-2020-0810). We will disseminate our results through an open-access peer-reviewed scientific publication and with a web-based clinical prediction tool.

Development and validation of self-monitoring auto-updating prognostic models of survival for hospitalized COVID-19 patients.

Clinical prognostic models can assist patient care decisions. However, their performance can drift over time and location, necessitating model monitoring and updating. Despite rapid and significant changes during the pandemic, prognostic models for COVID-19 patients do not currently account for these drifts. We develop a framework for continuously monitoring and updating prognostic models and apply it to predict 28-day survival in COVID-19 patients. We use demographic, laboratory, and clinical data from electronic health records of 34912 hospitalized COVID-19 patients from March 2020 until May 2022 and compare three modeling methods. Model calibration performance drift is immediately detected with minor fluctuations in discrimination. The overall calibration on the prospective validation cohort is significantly improved when comparing the dynamically updated models against their static counterparts. Our findings suggest that, using this framework, models remain accurate and well-calibrated across various waves, variants, race and sex and yield positive net-benefits.

Prognostic models for COVID-19 needed updating to warrant transportability over time and space.

BACKGROUND: Supporting decisions for patients who present to the emergency department (ED) with COVID-19 requires accurate prognostication. We aimed to evaluate prognostic models for predicting outcomes in hospitalized patients with COVID-19, in different locations and across time. METHODS: We included patients who presented to the ED with suspected COVID-19 and were admitted to 12 hospitals in the New York City (NYC) area and 4 large Dutch hospitals. We used second-wave patients who presented between September and December 2020 (2137 and 3252 in NYC and the Netherlands, respectively) to evaluate models that were developed on first-wave patients who presented between March and August 2020 (12,163 and 5831). We evaluated two prognostic models for in-hospital death: The Northwell COVID-19 Survival (NOCOS) model was developed on NYC data and the COVID Outcome Prediction in the Emergency Department (COPE) model was developed on Dutch data. These models were validated on subsequent second-wave data at the same site (temporal validation) and at the other site (geographic validation). We assessed model performance by the Area Under the receiver operating characteristic Curve (AUC), by the E-statistic, and by net benefit. RESULTS: Twenty-eight-day mortality was considerably higher in the NYC first-wave data (21.0%), compared to the second-wave (10.1%) and the Dutch data (first wave 10.8%; second wave 10.0%). COPE discriminated well at temporal validation (AUC 0.82), with excellent calibration (E-statistic 0.8%). At geographic validation, discrimination was satisfactory (AUC 0.78), but with moderate over-prediction of mortality risk, particularly in higher-risk patients (E-statistic 2.9%). While discrimination was adequate when NOCOS was tested on second-wave NYC data (AUC 0.77), NOCOS systematically overestimated the mortality risk (E-statistic 5.1%). Discrimination in the Dutch data was good (AUC 0.81), but with over-prediction of risk, particularly in lower-risk patients (E-statistic 4.0%). Recalibration of COPE and NOCOS led to limited net benefit improvement in Dutch data, but to substantial net benefit improvement in NYC data. CONCLUSIONS: NOCOS performed moderately worse than COPE, probably reflecting unique aspects of the early pandemic in NYC. Frequent updating of prognostic models is likely to be required for transportability over time and space during a dynamic pandemic.

Diagnostic Accuracy of Radioactive Iodine Seed Placement in the Axilla With Sentinel Lymph Node Biopsy After Neoadjuvant Chemotherapy in Node-Positive Breast Cancer.

Importance: Several less-invasive staging procedures have been proposed to replace axillary lymph node dissection (ALND) after neoadjuvant chemotherapy (NAC) in patients with initially clinically node-positive (cN+) breast cancer, but these procedures may fail to detect residual disease. Owing to the lack of high-level evidence, it is not yet clear which procedure is most optimal to replace ALND. Objective: To determine the diagnostic accuracy of radioactive iodine seed placement in the axilla with sentinel lymph node biopsy (RISAS), a targeted axillary dissection procedure. Design, Setting, and Participants: This was a prospective, multicenter, noninferiority, diagnostic accuracy trial conducted from March 1, 2017, to December 31, 2019. Patients were included within 14 institutions (general, teaching, and academic) throughout the Netherlands. Patients with breast cancer clinical tumor categories 1 through 4 (cT1-4; tumor diameter <2 cm and up to >5 cm or extension to the chest wall or skin) and pathologically proven positive axillary lymph nodes (ie, clinical node categories cN1, metastases to movable ipsilateral level I and/or level II axillary nodes; cN2, metastases to fixed or matted ipsilateral level I and/or level II axillary nodes; cN3b, metastases to ipsilateral level I and/or level II axillary nodes with metastases to internal mammary nodes) who were treated with NAC were eligible for inclusion. Data were analyzed from July 2020 to December 2021. Intervention: Pre-NAC, the marking of a pathologically confirmed positive axillary lymph node with radioactive iodine seed (MARI) procedure, was performed and after NAC, sentinel lymph node biopsy (SLNB) combined with excision of the marked lymph node (ie, RISAS procedure) was performed, followed by ALND. Main Outcomes and Measures: The identification rate, false-negative rate (FNR), and negative predictive value (NPV) were calculated for all 3 procedures: RISAS, SLNB, and MARI. The noninferiority margin of the observed FNR was 6.25% for the RISAS procedure. Results: A total of 212 patients (median [range] age, 52 [22-77] years) who had cN+ breast cancer underwent the RISAS procedure and ALND. The identification rate of the RISAS procedure was 98.2% (223 of 227). The identification rates of SLNB and MARI were 86.4% (197 of 228) and 94.1% (224 of 238), respectively. FNR of the RISAS procedure was 3.5% (5 of 144; 90% CI, 1.38-7.16), and NPV was 92.8% (64 of 69; 90% CI, 85.37-97.10), compared with an FNR of 17.9% (22 of 123; 90% CI, 12.4%-24.5%) and NPV of 72.8% (59 of 81; 90% CI, 63.5%-80.8%) for SLNB and an FNR of 7.0% (10 of 143; 90% CI, 3.8%-11.6%) and NPV of 86.3% (63 of 73; 90% CI, 77.9%-92.4%) for the MARI procedure. In a subgroup of 174 patients in whom SLNB and the MARI procedure were successful and ALND was performed, FNR of the RISAS procedure was 2.5% (3 of 118; 90% CI, 0.7%-6.4%), compared with 18.6% (22 of 118; 90% CI, 13.0%-25.5%) for SLNB (P < .001) and 6.8% (8 of 118; 90% CI, 3.4%-11.9%) for the MARI procedure (P = .03). Conclusions and Relevance: Results of this diagnostic study suggest that the RISAS procedure was the most feasible and accurate less-invasive procedure for axillary staging after NAC in patients with cN+ breast cancer.

Incremental prognostic value of acute serum biomarkers for functional outcome after traumatic brain injury (CENTER-TBI): an observational cohort study.

BACKGROUND: Several studies have reported an association between serum biomarker values and functional outcome following traumatic brain injury. We aimed to examine the incremental (added) prognostic value of serum biomarkers over demographic, clinical, and radiological characteristics and over established prognostic models, such as IMPACT and CRASH, for prediction of functional outcome. METHODS: We used data from the Collaborative European NeuroTrauma Effectiveness Research in Traumatic Brain Injury (CENTER-TBI) core study. We included patients aged 14 years or older who had blood sampling within 24 h of injury, results from a CT scan, and outcome assessment according to the Glasgow Outcome Scale-Extended (GOSE) at 6 months. Amounts in serum of six biomarkers (S100 calcium-binding protein B, neuron-specific enolase, glial fibrillary acidic protein, ubiquitin C-terminal hydrolase L1 [UCH-L1], neurofilament protein-light, and total tau) were measured. The incremental prognostic value of these biomarkers was determined separately and in combination. The primary outcome was the GOSE 6 months after injury. Incremental prognostic value, using proportional odds and a dichotomised analysis, was assessed by delta C-statistic and delta R2 between models with and without serum biomarkers, corrected for optimism with a bootstrapping procedure. FINDINGS: Serum biomarker values and 6-month GOSE were available for 2283 of 4509 patients. Higher biomarker levels were associated with worse outcome. Adding biomarkers improved the C-statistic by 0·014 (95% CI 0·009-0·020) and R2 by 4·9% (3·6-6·5) for predicting GOSE compared with demographic, clinical, and radiological characteristics. UCH-L1 had the greatest incremental prognostic value. Adding biomarkers to established prognostic models resulted in a relative increase in R2 of 48-65% for IMPACT and 30-34% for CRASH prognostic models. INTERPRETATION: Serum biomarkers have incremental prognostic value for functional outcome after traumatic brain injury. Our findings support integration of biomarkers-particularly UCH-L1-in established prognostic models. FUNDING: European Union's Seventh Framework Programme, Hannelore Kohl Stiftung, OneMind, Integra LifeSciences, and NeuroTrauma Sciences.

Does poor methodological quality of prediction modeling studies translate to poor model performance? An illustration in traumatic brain injury.

BACKGROUND: Prediction modeling studies often have methodological limitations, which may compromise model performance in new patients and settings. We aimed to examine the relation between methodological quality of model development studies and their performance at external validation. METHODS: We systematically searched for externally validated multivariable prediction models that predict functional outcome following moderate or severe traumatic brain injury. Risk of bias and applicability of development studies was assessed with the Prediction model Risk Of Bias Assessment Tool (PROBAST). Each model was rated for its presentation with sufficient detail to be used in practice. Model performance was described in terms of discrimination (AUC), and calibration. Delta AUC (dAUC) was calculated to quantify the percentage change in discrimination between development and validation for all models. Generalized estimation equations (GEE) were used to examine the relation between methodological quality and dAUC while controlling for clustering. RESULTS: We included 54 publications, presenting ten development studies of 18 prediction models, and 52 external validation studies, including 245 unique validations. Two development studies (four models) were found to have low risk of bias (RoB). The other eight publications (14 models) showed high or unclear RoB. The median dAUC was positive in low RoB models (dAUC 8%, [IQR - 4% to 21%]) and negative in high RoB models (dAUC - 18%, [IQR - 43% to 2%]). The GEE showed a larger average negative change in discrimination for high RoB models (- 32% (95% CI: - 48 to - 15) and unclear RoB models (- 13% (95% CI: - 16 to - 10)) compared to that seen in low RoB models. CONCLUSION: Lower methodological quality at model development associates with poorer model performance at external validation. Our findings emphasize the importance of adherence to methodological principles and reporting guidelines in prediction modeling studies.